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Abstract. In this paper, we consider reachability games over general hybrid systems, 
and distinguish between two possible observation frameworks for those games: either the 
precise dynamics of the system is seen by the players (this is the perfect observation 
framework), or only the starting point and the delays are known by the players (this is 
the partial observation framework). In the first more classical framework, we show that 
time-abstract bisimulation is not adequate for solving this problem, although it is sufficient 
in the case of timed automata. That is why we consider an other equivalence, namely the 
suffix equivalence based on the encoding of trajectories through words. We show that this 
suffix equivalence is in general a correct abstraction for games. We apply this result to o- 
minimal hybrid systems, and get decidability and computability results in this framework. 
For the second framework which assumes a partial observation of the dynamics of the 
system, we propose another abstraction, called the superword encoding, which is suitable 
to solve the games under that assumption. In that framework, we also provide decidability 
and computability results. 



Games over hybrid systems. Hybrid systems are finite-state machines equipped with a con- 
tinuous dynamics. In the last thirty years, formal verification of such systems has become 
a very active field of research in computer science, with numerous success stories. In this 
context, hybrid automata, an extension of timed automata [AD90, AD94], have been inten- 
sively studied [Hen95j [Hen96l, and decidable subclasses of hybrid systems have been drawn 
like initialized rectangular hybrid automata [Hen96]. More recently, games over hybrid sys- 
tems have appeared as a new interesting and active field of research since, among others, 
they correspond to a formulation of control problems, the counterpart of model checking 
for open systems, i.e., systems embedded in a possibly reactive environment. In this con- 
text, many results have already been obtained, like the (un) decidability of control problems 
for hybrid automata [HHM99 , or (semi-)algorithms for solving such problems [dAHMOl]. 

1998 ACM Subject Classification: F.3.1, F.4.1. 
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Given a system S (with controllable and uncontrollable actions) and a property ip, con- 
trolling the system means building another system C (which can only enforce controllable 
actions), called the controller, such that S \\ C (the system S guided by the controller C) 
satisfies the property (p. In our context, the property is a reachability property and our aim 
is to build a controller enforcing a given location of the system, whatever the environment 
does (which plays with the uncontrollable actions). 

O-minimal hybrid systems. O-minimal hybrid systems have been first proposed in [LPSOOj 
as an interesting class of systems (see [vdD98j for an overview of properties of o-minimal 
structures). They have very rich continuous dynamics, but limited discrete steps (at each 
discrete step, all variables have to be reset, independently from their initial values). This al- 
lows to decouple the continuous and discrete components of the hybrid system (see [LPSOO]). 
Thus, properties of a global o-minimal system can be deduced directly from properties of 
the continuous parts of the system. Since the introductory paper [LPSOO], several works 
have considered o-minimal hybrid systems |Dav99t lBMRT04j IBM051 IKV04j IKV06] . mostly 
focusing on abstractions of such systems, on reachability properties, and on bisimulation 
properties. 

Word encoding. In [BMRT04], an encoding of trajectories with words has been proposed 
in order to prove the existence of finite bisimulations for o-minimal hybrid systems (see 
also [BM05]). Let us mention that this technique has been used in [KV041 lKV06j in order 
to provide an exponential bound on the size of the finite bisimulation in the case of pfaffian 
hybrid systems. Let us also notice that similar techniques already appeared in the literature, 
see for instance the notion of signature in [ASYOlj . Different word encoding techniques have 
been studied in a wider context in [Bri07] . Recently in [KRS07j . the authors propose a new 
algorithm for counter-example guided abstraction and refinement on hybrid systems, based 
on use a word encoding approach. In this paper we use the so-called suffix encoding, which 
was shown to be in general too fine to provide the coarsest time-abstract bisimulation. 
However, based on this encoding, a semi-algorithm has been proposed in [Bri071 [Bri06] for 
computing a time-abstract bisimulation, and it terminates in the case of o-minimal hybrid 
systems. 

Contributions of this paper. In this paper, we focus on games over hybrid systems. We de- 
scribe two rather natural frameworks for such games, one assuming a perfect observation of 
the dynamics of the system, and another one assuming a partial observation of the dynamics. 
For the first framework, we use the above-mentioned suffix word encoding of trajectories for 
giving sufficient computability conditions for the winning states of a game. Time-abstract 
bisimulation is an equivalence relation which is correct with respect to reachability prop- 
erties on hybrid systems [AHLPOO] and with respect to control reachability properties on 
timed automata [AMPS98]. Here, we show that the time-abstract bisimulation is not cor- 
rect anymore for solving control problems on a general class of hybrid systems: we exhibit 
a system in which two states are time-abstract bisimilar, but one of the states is winning 
and the other is not. Using the suffix encoding of trajectories of |Bri07| . we prove that, 
in the perfect observation framework, two states having the same suffixes are equivalently 
winning or losing (this is a stronger condition than the one for the time-abstract bisimu- 
lation). We then focus on o-minimal hybrid games and prove that, under the assumption 
that the theory of the underlying o-minimal structure is decidable, the control problem can 
be solved and that winning states and winning strategies can be computed. Regarding the 
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partial observation framework, we provide a new encoding technique, the so-called super- 
word encoding, which turns out to be sound for the control under partial observation of the 
dynamics, and which allows to prove decidability and computability results similar to those 
in the perfect observation framework. 

Related work. The most relevant related works are those dealing with hybrid games [HHM99, 
dAHMOl]. However, the framework of these papers is pretty different from ours: 

(1) In their framework, time is considered as a discrete action, and once action "let time 
elapse" has been chosen, it is not possible to bound the time elapsing, which is quite 
restrictive. For instance, the timed game of Figure Q] is winning from (£q,x = 0) in 
our framework (the strategy is to wait some amount of time t £ [2,5] and to take the 
controllable action c), whereas it is not winning in their framework (once x is above 
5, it is no more possible to take the transition and reach the winning location £i, and 
there is no way to impose a delay within [2,5]). This yields significant differences in 
the properties: in their framework, game bisimulation is one of the tools for solving 
the games, and as stated by [HHM99, Prop. 1], the classical bisimulation tool is 
then sufficient to solve games. On the contrary, in our framework, the notion of 
bisimulation relevant to our model (time-abstract bisimulation) is not correct for 
solving games, as will be explored in this paper. 



(2) Our games are control games, they are thus asymmetric, which is not the case of the 
games in the above-mentioned works; in our framework, the environment is more 
powerful than the controller in that it can outstrip the controller and do an action 
right before the controller decides to do a controllable action. 
Let us also mention the paper [WT97J on control of linear hybrid automata. In [WT97] the 
author proposes a semidecision procedure for synthesizing controllers for such automata. 
No general decidability result is given in this paper. 

Plan of the paper. In Section [21 we recall results about finite games and bisimulation. In 
Section [3l we define the games over dynamical systems (for both perfect information and 
partial observation), and we show that time-abstract bisimulation is not correct for solving 
them. The word encoding techniques are presented in Section 0] and used in Section [5] 
to present a general framework for solving games over dynamical systems. We apply and 
extend these results in Section [6] for computing winning states and winning strategies in 
o-minimal games. In the paper, we often only develop technical details of the partial 
observation framework, which actually extends the perfect observation framework. 

Part of the results presented in this paper have been published in [BBC06I (the de- 
cidability of the control reachability problem and the synthesis of strategies for o-minimal 
hybrid systems). In this paper, we give full proofs of those results, and extend them to a 
natural partial observation framework. 




Figure 1: A simple game 
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2. Classical Finite Games 

In this section, we recall some basic definitions and results concerning bisimulations on 
a transition system (see [Acz88, Mil89, Cau95, Hcn95] for general references) and classical 
(untimed) games. 

2.1. Classical Games. We present here the definitions of the problem of control on a finite 
graph (also called finite game) and the notion of strategy (see [GTW 02 for an overview 
on games). These definitions are classical and will be extended to real-time systems in the 
next section. 

Definition 2.1. A finite automaton is a tuple A = (Q, Goal, E, 5) where Q is a finite set 
of locations, Goal C Q is a subset of winning locations, E is a finite set of actions, and 5 
consists of a finite number of transitions (q, a,q') G Q x E x Q. 

Definition 2.2. A transition system T = (Q, E, — >■) consists of a set of states Q (which 
may be uncountable), E an alphabet of events, and -^CQxSxQa transition relation. 

A transition (91,0,(72) G — >■ is also denoted by q\ A q^- A transition system is said 
finite if Q is finite. Note that a finite automaton canonically defines a transition system T4. 

A run of A is a finite or infinite sequence qo gi — ^> ... of the transition system T4. 
Such a run is said winning if % G Goal for some i. If p is a finite run qo . . . g n 

we define last(p) = q n . We note RunSj(^l) the set of finite runs in A. 

Definition 2.3. A finite game is a finite automaton (Q, Goal, E, S) where E is partitioned 
into two subsets E c and E u corresponding to controllable and uncontrollable actions. 

We will consider control games. Informally there are two players in such a game: the 
controller and the environment. The actions of E c belong to the controller and the actions 
of Ti u belong to the environment. At each step, the controller proposes a controllable 
action which corresponds to the action he wants to perform; then either this action or an 
uncontrollable action is done and the automaton goes into one of the next states^- In the 
sequel, we will only consider reachability games : the controller wants to reach the Goal 
states and the environment wants to prevent him from doing so. 

Definition 2.4. A strategy is a partial function A from RunS/(„4) to E c such that for all 
runs p G RunS/(„4), if X(p) is defined, then it is enabled in last(p). 

Let p = qo — ^> qi ... be a run, and set for every i, pi the prefix of length i of p. The 
run p is said compatible with a strategy A when for all i, a^+i = \(pi) or a^+i G E u . A run 
p is said maximal w.r.t. a strategy A if it is infinite or if X(p) is not defined. 

A strategy A is winning from a state q if all maximal runs starting in q compatible with 
A are winning. 



There may be several next states as the game is not supposed to be deterministic, and we assume that 
the environment chooses the next state in case there are several. 
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2.2. Bisimulation. We recall now the definition of bisimulation for transition systems: 

Definition 2.5 ([Mil89 j ICau95| ). Given a transition system T = (Q, £,—)>), a bisimulation 
for T is an equivalence relation ~CQxQ such that Vqi,q[,q2 € Q, Va € K, 



Moreover, if V is a partition of Q and if ~ respects V {i.e., q £ P and q ^ q' with P £ V 
implies g' £ P), we say that ~ is compatible with P. 

2.3. Game and Bisimulation in the Untimed Case. In the untimed framework, bisim- 
ulation is a commonly used technique to abstract games: bisimilar states can be identified 
in the control problem. This is stated in the next folklore theorem, for which we provide a 
proof. 

Theorem 2.6. Let A = (Q, Goal, E, 5) be a finite game, q,q' £ Q and ~ a bisimulation 
compatible with Goal. Then, there is a winning strategy from q iff there is a winning strategy 
from q' . 

Proof. Assume that ~ is a bisimulation relation compatible with Goal and such that q ~ q' . 
Assume furthermore that A is a winning strategy from q. We will define a strategy A' that 
will be winning from q'. To do that we will map finite runs starting in q' to finite runs 
starting in q, so that A' will mimick A through this mapping. We note / this mapping, and 
start by setting /(</) = q. We then proceed inductively as follows. If X(f(g')) is defined, 

we set X'(g') = X(f(g')) and for every run g' > (f (which is compatible with A') there 

is a run f(g') q which is compatible with A and such that q ~ <f . We then define 

f(g' — <f ) = f(g') ^> q- The strategy A' is winning from q' since ~ is compatible 
with Goal. □ 

This theorem remains true for infinite-state discrete games [HHM99, dAHMOli and 
can be used to solve them: if an infinite-state game has a bisimulation of finite index, the 
control problem can be reduced to a control problem over a finite graph. Real-time control 
problems cannot be seen as classical infinite-state games because of the special nature of 
the time-elapsing action, which does not belong to one of the players. It seems nevertheless 
natural to try to adapt the bisimulation approach to solve real-time control problems. 



3.1. Dynamical Systems. Let Ad be a structure. When we say that some relation, subset 
or function is definable, we mean it is first-order definable in the structure A4. A general 
reference for first-order logic is [Hod97]. We denote by Th(A / () the theory of A4. In this 
paper we only consider structures M. that are expansions of ordered groups, we also assume 
that the structure A4 contains two symbols of constants, i.e., M. = {M, + , 0, 1, <, . . .) where 
+ is the group operation and w.l.o.g. we assume that < 1. 

Definition 3.1. A dynamical system is a pair (A4,j) where: 

• M. = (M, +, 0, !,<,.. .) is an expansion of an ordered group, 




{ and A q 2 ) ( 3q' 2 q 2 ~ q' 2 and q[ A q' 2 




3. Games over Dynamical Systems 
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• 7 : V\ x V — > V-z is a function definable in M (where V\ C M fcl , V C M and 

The function 7 is called the dynamics of the system. 

Classically, when M is the field of the reals, we see V as the time, V± as the input space, 
V\ x V as the space-time and V 2 as the (output) space. We keep this terminology in the 
more general context of a structure M. 

The definition of dynamical system encompasses a lot of different behaviors. Let us 
first give a simple example, several others will be presented later. 

Example 3.2. We can recover the continuous dynamics of timed automata (see [A I )!) 1 ) . 
In this case, we have that M = (R, <, +, 0, 1) and the dynamics 7 : W 1 x [0, +oo[— > W 1 is 
defined by 7(27, . . . ,x n ,t) = (xi + t, . . . ,x n + t). 

Definition 3.3. If we fix a point x G V%, the set T x = {7(2, t) \ t G M + } C V% is called the 
trajectory determined by x. 

We define a transition system associated with the dynamical system. This definition 
is an adaptation to our context of the classical continuous transition system in the case of 
hybrid systems (see [LPS00] for example). 

Definition 3.4. Given (A4,^y) a dynamical system, we define a transition system T 7 = 
(Q, E, — Ky) associated with the dynamical system by: 

• the set Q of states is V 2 ; 

• the set E of events is M + = {r G M | r ^ 0}; 

• the transition relation yi A 7 2/2 is defined by: 

3x e Vi, 3ti,i 2 G M + such that ti ^ i 2 , 

7(x, ti) = yi, j(x, t 2 ) = y2 and t = t 2 - h 

3.2. M-Games Under Perfect Observation. In this subsection, we define .M-automata, 
which are automata with guards, resets and continuous dynamics definable in the M- 
structure. We then introduce our model of dynamical game which is an Af-automaton with 
two sets of actions, one for each player; we finally express in terms of winning strategy the 
main problem we will be interested in, the control problem in a class C of .A/f-automata under 
perfect observation. The partial observation framework will be discussed in Subsection 13.31 

Definition 3.5 (.M-automaton). An Ai-automaton A is a tuple (M, Q, Goal, E, 5, 7) where 
M. = (M, +, 0, 1, <, . . .) is an expansion of an ordered group, Q is a finite set of locations, 
Goal C Q is a subset of winning locations, E is a finite set of actions, 5 consists in a finite 
number of transitions (q,g,a,R,q') G Q x 2 V2 x E x (V2 — > 2 V2 ) x Q where g and R are 
definable in Ai, and 7 maps every location q G Q to a dynamics 7 g : Vi x V — ¥ 

We use a general definition for resets: a reset R is indeed a general function from V2 to 
2 Vl , which may correspond to a non-deterministic update. If the current state is (q, y) the 
system will jump to some (q',y') with y' G R{y)- 

An .M-automaton A = (Ai, Q, Goal, E, S, 7) defines a mixed transition system T_4 = 
(S, T, — >) where: 



'We use these notations in the rest of the paper. 
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• the set S of states is Q x V2; 

• the set T of labels is M+ U S, (where M+ = {r G M | r ^ 0}); 

• the transition relation (q, y) A- (q',y') is defined when: 

— e G S, and there exists (q,g,e,R,q') £ <5 with y G y and y' G i?(y), or 

— e G M + , q = q' , and y A 7<7 y' where 7 g is the dynamic in location q. 

In the sequel, we will focus on behaviors of .M-automata which alternate between 
continuous transitions and discrete transitions. 

We will also need more precise notions of transitions. When (q, y) — > (q, y') with 
r G M + , this is due to some choice of (x,t) G V\ X V such that ^y q (x,t) = y. We say 
that (q,y) —t x ,t (Q,y') if 7q( x ,t) = V and 7g(x,i + t) = y'. To ease the reading of the 
paper, we will sometimes write (q,x,t,y) A (q, x,t + r, y') for (g, y) — K^t (q,y'). We say 
that an action (r, a) G M + x E is enabled in a state (q, x, t, y) if there exists (q 1 , x', t', y') 
and (q",x",t",y") such that (q,x,t,y) -H> (q' ,x' ,t' ,y') A- (q" , x" ,t" ,y"). We then write 

(g,M,y)^teVV',y"). 

A run of is a finite or infinite sequence (go, ^0; yo) — ; — > (<li,%l,ti,yi) ■ ■ ■ Such a 
run is said winning if qi G Goal for some i. 

We note RunSj(^l) the set of finite runs in A. If p is a finite run (qo, xq, to, yo) Tl,ai > 
. . . T "' a "> (q n ,x n ,t n ,y n ) we define /ast(p) = (q n , x n ,t n ,y n ). 

Definition 3.6 (.M-game). An M-game is an .M-automaton (A4 , Q, Goal, S, 5, 7) where S 
is partitioned into two subsets S c and S u corresponding to controllable and uncontrollable 
actions. 

Definition 3.7 (Strategy). A strateg^ is a partial function A from RunSj(^4) to M + x S c 
such that for all runs p in RunSj(*A), if X(p) is defined, then it is enabled in last(p). 

The strategy tells what is to be done at the current moment: at each instant it tells 
what delay we will wait and which controllable action will be taken after this delay. Note 
that the environment may have to choose between several edges, each labeled by the action 
given by the strategy (because the original game is not supposed to be deterministic). 

A strategy A is said memoryless if for all finite runs p and p', last(p) = last(p') implies 
X(p) = X(p'). Let p = (qo,xo,to,yo) Tl,ai > ... be a run, and set for every i, pi the prefix of 
length i of p. The run p is said consistent with a strategy A when for all i, if \(pi) = (t, a) 
then either Tj + i = r and cn + i = a, or Ti+i ^ r and a^+i G S u . A run p is said maximal 
w.r.t. a strategy A if it is infinite or if X(p) is not defined. A strategy A is winning from 
a state (q,y) if for all (x,t) such that 7(x,i) = y, all maximal runs starting in (q,x,t,y) 
compatible with A are winning. The set of winning states is the set of states from which 
there is a winning strategy. 

We can now define the control problems we will study. 

Problem 3.8 (Control problem under perfect observation in a class C of .M-automata) . 
Given an A4-game A G C, and a definable initial state (q, y), determine whether there exists 
a winning strategy in A from (q,y). 



'In the context of control problems, a strategy is also called a controller. 



8 



P. BOUYER, T. BRIHAYE, AND FABRICE CHEVALIER a 



Problem 3.9 (Controller synthesis under perfect observation in a class C of .M-automata) . 
Given an .M-game A £ C, and a definable initial state (q, y), determine whether there exists 
a winning strategy, and compute such a strategy if possible^ 

Example 3.10. Let us consider the .M-game A = (M, Q, Goal, S, 5, 7) (depicted in Fig. [5]) 
where M. = (R, +, ■, 0, 1, <, sin, cos), Q = {qi, q 2 , 93}, Goal = {q 2 }, S = S c U S u where 
S c = {c} (resp. S u = {u}) is the set of controllable (resp. uncontrollable) actions. The 
dynamics in qi, 7 gi : R 2 x [0, 2tt] x R ->• R 2 is defined as follows. 



-y qi (xi, x 2 ,9,t) 



(i.cos(6»),i.sin(0)) if (x t ,x 2 ) = (0,0), 

(21 +t.xi,a;2 +t.x 2 ) if (xi,x 2 ) 7^ (0,0). 

We associate with this dynamical system the partition V = {A, B,C} where A = {(0,0)}, 
B = {(flcos(fl),fl sin(fl) ) I < 9 ^ 2vr} and C = R 2 \ (A U C). Let us call piece B the 
spiral (see Figure 2(b)). The guard gs corresponds to -B-states (i.e., points on the spiral) 



and the guard gc corresponds to C-states (points not on the spiral and different from the 
origin). In this example, the point (gi, (0,0)) is a winning state. Indeed a winning strategy 





(a) The .M-game A (b) Dynamics in q\ 

Figure 2: Time-abstract bisimulation does not preserve winning states 

(f , c) where c consists in taking the transition leading to state 



is given by A (91, 0, 0, 9, t 
q 2 (which is winning). 



3.3. A4-Games Under Partial Observation. Subsection l3.21 we have assumed that from 
a given point, the environment chooses the continuous trajectory followed by the game, and 
the controller reacts accordingly. In this section, we consider partial observation of the 
dynamics: the trajectory is not known by the controller, and its strategy may depend only 
on the current point. In particular, this framework naturally models drift of clocks where 
the slopes of the clocks lies within an interval |Pur98} IALM 05 . Note that our partial 
observation assumption concerns the dynamics of the system, not the actions which are 
performed. This has to be contrasted with the notion of partial observation studied in the 
framework of finite systems in |AVW03] or in the context of timed systems in [BDMP03 

In this definition, 'compute a strategy' means 'give a formula for the strategy'. In particular, a strategy 
which is computable is definable in the theory. 
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where the partial observation assumption concerns actions which are done, and not the 
dynamics (indeed, in these models, there is no real choice for the dynamics; It is completely 
determined by the point in the state-space). In order to formalize our partial observation 
framework, we need to adapt notions such as strategy in this new setting. First, we define 
what we call observation of a given run. 

Definition 3.11 (Observation of a run). Let p = (qo,xo,to,yo) n ' \ ... T "'°"> {q n ,x n ,t n ,y n ) 
be a finite run. The observation of p, denoted obs(p) is the sequence (qo,yo) T1,ai > . . . T "'°"> 

(q n ,yn)- 

Definition 3.12 (Strategy under partial observation). A strategy A is said under partial 
observation if for all finite runs p, p', obs(p) = obs(p') implies X(p) = X(p'). 

All other notions, like memoryless strategies, consistency, winning strategies, winning 
states, etc... naturally extend in this new context. In this setting, we will consider the two 
following problems. 

Problem 3.13 (Control problem under partial observation in a class C of .M-automata). 
Given an .M-game A G C, and a definable initial state (q, y), determine whether there exists 
a winning strategy under partial observation in A from (q, y). 

Problem 3.14 (Controller synthesis under partial observation in a class C of .M-automata) . 
Given an .M-game A G C, and a definable initial state (q, y), determine whether there exists 
a winning strategy under partial observation in A from (q,y), and compute such a strategy 
if possible. 

Example 3.15. We consider again the spiral example (Example 13. lOj) . We showed that 
under perfect observation this .M-game has a winning strategy in (</i,(0, 0)) given by 
\(qi, 0, 0, 0, t) = (|,c). Note that this strategy depends on the precise trajectory (parame- 
ter 9). Moreover, one can show that there is no winning strategy under partial observation 
for this game: such a strategy may only depend on the current point, and in this precise 
example, whatever action (r, a) the controller proposes in (q\, (0,0)), there is a trajectory 
which reaches a bad state (i.e., points on the spiral) before r. 

The previous example shows that some games can be winning under perfect observation 
whereas they are not winning under partial observation. Nevertheless, considering a new 
dynamics which will roughly inform the controller of the current trajectory, we can see the 
perfect observation control problem as a special case of the partial observation framework. 
This is stated by the following proposition : 

Problem 3.16. Given an .M-game Ai and a state (q, y) of Ai, we can effectively con- 
struct an .M-game A2 and a state (q',y') of A2 such that there exists a winning strategy 
under perfect observation in Ai from (q, y) iff there exists a winning strategy under partial 
observation in Ai from (q',y'). 

Proof. Let Ai = (M, Q, Goal, S, S, 7) where 7 : V x X V ->• V%. We define V 2 ' = {(x,t,y) G 
Vi X V X V2 I 7(x, t) = y} and for q G Q, 7' : V\ X V — > V 2 ' such that 7^(2, t) = (x, t, j q (x, t)). 
The dynamics 7' behaves exactly like 7 but "gives" to the controller the current trajectory 
as this information is stored in the state space V 2 '. 

We then use Ai = (A4, Q, Goal, S, 6', 7'), where 5' is the transition relation 5 adapted to 
the new states V 2 K . if (qi,g, a, R, q 2 ) G 5 then (qi,g', a, R', (72) G 5' where g' = {(x, t, j(x, t)) \ 
j(x,t) G g} and for all (x,t) G V\ x V, R'(j(x,t)) = {(x' , t', j(x', t')) \ j(x',t') G R(>y(x,t))}. 
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W.l.o.g. we can suppose that there exists a unique (xo, in) € V\ x V such that 7(xn, to) = 
y (if necessary, we add a location with constant continuous dynamics pointing to the actual 
location of y). Then there exists a winning strategy under perfect observation in Ai from 
(q, y) iff there exists a winning strategy under partial observation in A2 from (q, (xq, to,y)). 



From the above proposition we get that any definability, decidability, etc result in the 
partial observation framework will hold in the perfect observation framework. 

3.4. .M-Games and Bisimulation. Time-abstract bisimulation [H en95l [Dav99[ [SHLPOOj 
is a sufficient behavioral relation to check reachability properties of hybrid systems, and 
in particular of .M-automata [Bri07]. Moreover, it has been shown that it is also a suf- 
ficient behavioral relation in order to solve control problems in the framework of timed 
automata [AMPS98J. However, when considering wider classes of hybrid systems, we will 
see that this tool is not sufficient anymore for solving control problems in the perfect ob- 
servation framework. 

Definition 3.17. Given a mixed transition system T = (S,T, — >), a time-abstract bisimu- 
lation for T is an equivalence relation ~ C S x S such that Vgi, q'i, qi 6 S, the two following 
conditions are satisfied: 



Example 3.18. In this example, we assume a perfect observation framework. Let us 
consider the .M-game A = (M,Q, Goal, S, 6, 7) where M = (R, <, +, 0, 1, =2} (=2 denotes 
the "modulo 2" relation), Q = {q%, q2, (73}, Goal = {92}) S = S C UE U where S c = {c} (resp. 
S u = {u}) is the set of controllable (resp. uncontrollable) actions. The dynamics in qi, 
j gi : R + x {0,1} xl+->K+x {0,1} is defined as j qi (xi, x 2 , t) = {x x +t,x 2 ). 



□ 




) 




x 2 = 



X 2 = 1 



A B C B C 



A C B C B 



(a) The A4-game A 



(b) Dynamics in q\ 



Figure 3: Time-abstract bisimulation does not preserve winning states 



We consider the partition depicted on Figure [3(b)] The guard gc is satisfied on C-states 
and the guard gs is satisfied on £>-states. Note that this partition is compatible with Goal 
and w.r.t. discrete transitions. 
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In this game, the controller can win when it enters a C-state by performing action c and 
it loses when entering a .B-state because it cannot prevent the environment from performing 
a u and going in the losing state #3. 

It follows that the state s± = (qi, (0, 1)) is losing, whereas the state S2 = (qi, (0, 0)) is 
winning. However, the equivalence relation induced by the partition {A,B,C} is a time- 
abstract bisimulation: the two states si and S2 are thus time-abstract bisimilar, but not 
equivalent for the game. It follows that time-abstract bisimulation is not correct for solving 
control problems, in the sense that a time-abstract bisimulation cannot always distinguish 
between winning and losing states. 

Problem 3.19. Let Ai be a structure and A an .M-game. A partition respecting Goal 
and inducing a time-abstract bisimulation on Q x V2 does not necessarily respect the set of 
winning states of A. 

4. The Suffix and the Superword Abstractions 

In this section we explain how to encode symbolically trajectories of dynamical systems 
with "words" . We will present two different encodings (or abstractions) depending on the 
observation framework (perfect or partial) we assume. 

4.1. Perfect Observation and the Suffix Abstraction. In this subsection, we review 
the word encoding technique introduced in [BMRT04] in order to study o-minimal hybrid 
systems. We focus on the suffix partition introduced in [Bri07| . This encoding will be 
suitable in order to study control reachability problem in the perfect observation framework 
(see Subsection l5.3p . We first explain how to build words associated with trajectories. Given 
a dynamical system (M. , 7) and a finite partition V of V2 , given x G V± we associate a word 
with the trajectory T x = {j(x, t) | t G V} in the following way. We consider the sets 
{t G V I j(x, t) G P} for P G V. This gives a partition of the time V. In order to define 
a word on V associated with the trajectory determined by x, we need to define the set of 
intervals T x = {J | I is a time interval or a point and is maximal for the property "3P G 
V, Vi G /, j(x, t) G P"}. For each x, the set T x is totally ordered by the order induced 
from M. This allows us to define the word on V associated with the trajectory T x denoted 
u x . 

Definition 4.1. Given x G V\, the word associated with T x is given by the function uj x : 
F x — > V defined by uj x (I) = P, where / G T x is such that Vi G /, *y(x, t) G P. 

The set of words associated with (A4,7) over V gives in some sense a complete static 
description of the dynamical system (A^,7) through the partition V. In order to recover 
the dynamics, we need further information. 

Given a point x of the input space V%, we have associated with x a trajectory T x 
and a word oj x . If we consider (x,t) a point of the space-time V\ x V, it corresponds to 
a point -f(x,t) lying on T x . To recover in some sense the position of j(x,t) on T x from 
ui x , we associate with (x,t) a suffix of the word ui x denoted 0Jr x ,t)- The construction of 
cj^ j) is similar to the construction of oj x , we only need to consider the sets of intervals 
F(x,t) = {1 n {? G V I f > t} I / G T x ). 

Let us notice that given (x, t) a point of the space-time V\ X V there is a unique suffix 
u)r Xit \ of lo x associated with (x,t). Given a point y G V2 it may have several (x,t) such that 
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j(x,t) = y and so several suffixes are associated with y. In other words, given y E V2, the 
future of y is non-deterministic, and a single suffix W( x ,t) is thus not sufficient to recover 
the dynamics of the transition system through the partition V. To encode the dynamical 
behavior of a point y of the output space V2 through the partition V, we introduce the 
notion of suffix abstraction (called suffix dynamical type in [Bri071 IBri06| ) of a point y 
w.r.t. V. 

Definition 4.2. Given a dynamical system (.M,7), a finite partition V of V2, a point 
y £ V2, the suffix abstraction of y w.r.t. V is denoted Suf-p(y) and defined by Suf-p(y) = 

W(x,t) I l(x,t) = y}. 

This allows us to define an equivalence relation on V2. Given yi, y2 € V2, we say 
that they are suffix- equivalent if and only if Sufp(yi) = Suf-p^)- We denote Suf (V) the 
partition induced by this equivalence, which we call the suffix partition w.r.t. V . We say 
that a partition V is suffix-stable if Suf('P) = V (it implies that if y\ and 2/2 belong to the 
same piece of V then Sllfp(yi) = Sufp^))- 

To understand the suffix abstraction technique, we provide several examples. 

Example 4.3. We start with example 13.101 The suffix abstraction in (0,0) is composed of 
a unique suffix ACBC because any trajectory leaving (0, 0) crosses exactly once the spiral 
at some point. By looking at Fig. [2] one can convince oneself that the suffixes associated 
with the other points of the plane are given by suffixes of ACBC; for instance, the points 
lying on the spiral (the piece B) have suffix BC . 

Example 4.4. We first consider a two dimensional timed automata dynamics (see Exam- 
ple In this case we have that j(x%,X2,t) = (x\ + t, X2 + t). We associate with this 
dynamics the partition V = {A, B} where B = [1, 2] 2 and A = R 2 \ B. In this example the 
suffix partition is made of three pieces, which are depicted in Figure HI 



x 2 



A 



BA 



ABA / 



Figure 4: Suffixes for the timed automata dynamics 



The suffix abstraction allows to encode more sophisticated continuous dynamics than 
the previous suffix encoding of a trajectory. In the next example we recover in some sense 
the continuous dynamics of rectangular automata [HKPV98] , which requires to use the suffix 
abstraction (some of the points do not have a unique suffix). 

Example 4.5. We consider the dynamical system (At, 7) where A4 = (R, +, •, 0, 1, <) and 
7 : R 2 x [1, 2] xR + — > R 2 is defined by 7(^1, X2,p, t) = (x\+t, x 2 +p-t). We associate with this 
dynamical system the partition V = {A,B,C} where B = [2,5] x [3,4], C = [3,5] x [1,2] 
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and A = R 2 \ (B U C) (see Figure 5(a)). Let us focus on the suffix abstractions of the 
two points yi = (1,2.5) and 1/2 = (2,0.5). We have that Sufp(yx) = {A, ABA} and 
Sufp(y2) = {ABA, AC ABA}. Though several points have several possible suffixes, the 



partition induced by the suffix abstraction is finite and illustrated in Figure 5(b) 




{A,ABA} < 




(a) The dynamics (b) The suffix partition 

Figure 5: A rectangular dynamics 



4.2. Partial Observation and the Superword Abstraction. The suffix-partition pro- 
posed in Subsection 14.11 is not suitable for the partial observation framework. We will 
intuitively convince the reader of this fact. Let (M, 7) be a dynamical system, y be a point 
of V2 and V be a partition of V2. Since several trajectories cross the point y, there exist 
several y' such that y -4- y\ for some r G M + . In the partial observation framework, the 
controller does not know which trajectory will be chosen by the environment and have to 
choose a pair (r, c) independently. In particular, starting from y, one can potentially be 
in several different pieces of V after r time units. The notion of suffix abstraction is not 
sufficient in order to capture these behaviors, that is why we now associate a word u y on 
2^ with a given y G V2 . We will see in Subsection 15.21 that this new encoding is suitable in 
order to study control reachability problem in the partial observation framework. In order 
to define the word on 2^ associated with y G V2, we need to introduce further definitions. 

Definition 4.6. Let y be a point of V2 and r be a time in M + . 

T v (t) = {P g V I 3x G M kl 3t G M j(x,t) = y and j(x,t + r) G P}. 

The set F y {r) represents the set of pieces that we have potentially reached after r time 
units when starting from y. 

Definition 4.7. Let y be a point of Vi- 

T y ={l I I is a time interval and is maximal for the property 

3S G 2 V Vr G / F y {r) = S) 

For each y G V2, the set T y exactly consists of the connected components of the sets 
{r G M+ I T y {j) = S}, for S G 2 V . We can now define the superword Sup-p(y) associated 
with a given y G Vi- 
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Definition 4.8. Let (Al, 7) be a dynamical system, y be a point of V2, and V be a partition 
of V2. T/ie superword associated with y is given by the function Sup.p(y) : J- y ^ 2 V defined 
by: 

Sup^, (?/)(!) = S where I £ J- y is such that Vr G I ^(r) = S 1 . 

Let us notice that given (7W,7) a dynamical system, V a partition of V2, and y a 
point of V2, there exists a unique superword Sup-p(y) associated with y. If (.M,7) is a 
dynamical system and V a finite partition of V2, we write Sup('P) for the partition induced 
by superwords. We say that a partition V is superword- stable if Sup('P) = V. Let us 
illustrate this new notion on examples. 

Example 4.9. Let us consider the three dynamical systems depicted on Figures [H In 
the three cases, the dynamical system consists of two trajectories exiting the point y{. 
What differs in the three systems is the way the partition V = {A, B,C} is crossed. We 
are interested in the superword associated with j/j. For the two first dynamical systems 
we have that Sup-p(yi) = Sup-p(i/2) = {A}{B,C}, and for the last one we have that 
Sup P (y 3 ) = {A}{B,C}{B}{B,C}{C}{B,C}. 



B C B B B C B 

A / 1 1 A / A / 1 1 



l / 1 1 a y~ 

yi \ , , V2 \ 2/3 \ — , , 

C B C C C B C 

(a){A}{B,C7} (b) {A}{B,C} (c) {A}{B, C}{B}{B, C}{C}{B, C} 

Figure 6: Suffix and superword are not comparable 

Let us notice that the notions of suffix abstraction and superword abstraction are in- 
comparable. To illustrate this fact, let us consider again the three dynamical systems of 
Figure [6l We have that Sup-p(yi) = Supp(y2) 7^ Supp(y3). Let us now consider the suffix 
abstractions of these points: 

Suf(yi) = {ABCB, ACBC} ; Su% 2 ) = {AB,AC} ; Suf(y 3 ) = {ABCB, ACBC}. 

This shows that the superword abstraction can distinguish between y\ and 2/3, but cannot 
distinguish between y\ and 7/2 > although the suffix abstraction can distinguish between y\ 
and 2/2> but cannot distinguish between y\ and 2/3. 



5. Solving an .A4-Game 

In this section we first present a general procedure to compute the set of winning states 
for an .M-game under partial observation. We then show that if a partition is superword- 
stable, the procedure can be performed symbolically on pieces of the partition. The proce- 
dure described is not always effective and we will later point out specific .M-structures for 
which each step of the procedure is computable. By Proposition 13.161 we know that the 
perfect observation control problem can be seen as a special case of the partial observation 
framework; however at the end of this section, we explain how the suffix partition can be 
used in order to directly solve the perfect observation control problem. 
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5.1. Controllable Predecessors under Partial Observation. As for classical reacha- 
bility games [GTW 02] , one way of computing winning states is to compute the attractor of 
goal states by iterating a controllable predecessor operator. Let A = (M, Q, Goal, S, 5, 7) be 
an A4-game. For W C Q x V2, a E S c and u E S u we first define the notion of controllable 
discrete predecessors. For every a E E = S c U £ u , we have 

a is enabled in (q,y), 
and V(g',y') E Q x V 2 , 



Pred a (W) 



(g, y) E Q x l/ 2 



The intuition of this operator is the following: a state is in Pred a (W) if action a can be done 
from (q,y), and whichever transition is taken leads to a state in W (action a ensures W in 
one step). We also define cPred(W) = (J Pred c (W) and uPred(VF) = \J Pred a (W). 

As for timed and hybrid games [AMPS98, HHM99], we also define a safe time predeces- 
sor of a set W w.r.t. a set W , that is specific to the partial observation framework: a state 
(q,y) is in time-PredpartiaKVF, W) if a delay r can be chosen such that for all trajectories 
starting from (q, y), one can let r time units pass avoiding W' and then reach (q 1 , y') E W. 
Formally the operator time-Predp ar tj a i is defined as follows: 

3t E M+, V(M) E Vl x y s.t. 
7g(x,t) = y, and ^ Xjt (g',y') 

implies ((q',y') E W and Post|'* 

i < t' < i + r}. 



time-Pred P artiai(^,^') 



(9, y) € Q x y 2 



where Postj^ 



{7,(*,0 



as: 



The controllable predecessor operator under partial observation 7Tp ar tj a i is then defined 
partial (W) = WU (J time-Predp artia i(Pred a (W), uPred(V7)). 

Remark 5.1. Note that the operator vr part j a | is definable in any expansion of an ordered 
group. Hence, if W is definable, so is vrp art j a i(VF). 

Example 5.2. We first illustrate the computation of the operator 7Tp ar tj a i on Example 13. 101 
(see page[8|). In this case, vr^partial does not induce a winning strategy from (q%, (0,0)) under 
partial observation. Setting W = Goal x V2 = {(72} x V2, we have that 7r par tj a i(W) does not 
contain the point (qi, (0, 0)) because there is no uniform choice for a positive delay r before 
taking action c so that the spiral (area B) can be avoided. Notice however that 7r pa rtial (W) 
is not empty because it includes all points different from (qi, (0,0)) (from which there is a 
unique trajectory). 

Remark 5.3. Note also that due to the partial observation assumption, in the definition 
of 

^partial) the action a for controlling the system has to be chosen before choosing the delay 
r. Indeed, the controller does not know which precise trajectory will be chosen by the 
environment, in particular, action a should be available after time r independently of the 
choice of trajectory made by the environment. This is illustrated in the next example. 



Example 5.4. Let us consider the A4-game A depicted on Figure 7(a) where Goal = 
{52,93} and where ci,C2 E S c are distinct controllable actions. The dynamics in q\ is 
depicted on Figure |7(b)[ roughly speaking, it consists of of two trajectories exiting the 



16 



P. BOUYER, T. BRIHAYE, AND FABRICE CHEVALIER a 



point y. perfect observation from y; indeed depending on the trajectory we are following, 
we will either play (r, c{) or (r, C2), for some well-chosen r G M + . However, there is no 
winning strategy under partial observation from y. Although we can find r G IR + such that 
a controllable action will be (safely) available (from y) after r time units, we are unable to 
tell which controllable action will be taken. 

In fact if W = Goal x V 2 we have that vr partia |(M^) = {(qi,z) \ z G V2\{y}}. Indeed if 
(qi,z) 7^ (qi,y), the controller can deduce the trajectory from the current state and choose 
its action accordingly. 




(a) The M-g&me A (b) Dynamics in qi 



Figure 7: 

The next proposition states the soundness of this operator for computing winning states 
in the games under a partial observation hypothesis. 

Problem 5.5. Let A = {M., Q, Goal, S, 5, 7) be an .M-game. If there exists n G N s.t. 

partial ( Goal ) = ^partial ( Goal ) then ^partial ( Goal ) = ^partial ( Goal ) is the set of winning states 
of A under partial observation. 

Proof. We first prove that if (q, y) G 7r* artja |(Goal) then there exists a winning strategy under 
partial observation from (q,y). To this aim, we define a memoryless winning strategy from 
any (q, y) G 7r* artja |(Goal). By notation misuse, we define the strategy A on states (q,y) 
instead of executions. 

We define a strategy A on all sets IJosgi^fc ^partial ( Goa ') by induction on k, and prove 
that it is a winning strategy. If k = 0, we assume A is defined nowhere, it is thus winning 
from all states in Goal. 

Suppose now that A is already defined on W = Uo^i^fc ^partialC^ 03 ') ana - ^ s wrnnm g 
on these states. We now define A on np ar i\ a \(W) . Let (q,y) G Q x Vi\ if (q,y) G W, A is 
already defined; if (q,y) G 7Tp art j a i(W0 \ W, then we know that there exists a G S c with 
(q,y) G time-Predpartiai (Pred a (W),uPred(W)). There exists r G M+ with (r, a) enablecfl 
in (q,y) suc h that for e very (x,t) if J q (x,t) = y, then (q,y) ^ x>t (q',y'), W ,y') G W and 
Post^ +r j C uPred(W A ). We set X(q,y) = (r, a) and show that this is a winning choice. 

We show by induction on k that A is winning for each state of W = Uo^i^fc ^partial (G° a Q- 
This is immediate for k = 0. Suppose now that the result is true for k and let (q, y) G 
^partial (W). Let p = (q,x,t,y) T1,ai ) (qi,xi,ti,yx) T2 ' a2 ) ... be an execution compatible 
with A. We have that either r\ = r and a± = a, in which case (qi,yi) G W, or t\ ^ r 
and a\ G S u , in which case 

(q,v) -^x,t (q',y') (qi, Vi) with (q',y') £ uPred(VF) so 
(<?i>yi) £ W. In both cases, (qi,yi) G W so by induction hypothesis, p is winning. 

^We say that (r, a) £ M + x E is enabled in (q, y) if there exists (x,t) £Vi xV such that ~f(x,t) = y and 
(r, a) is enabled in (q,x,t,y). 
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We now show that if there exists a strategy under partial observation A winning from 
(q,y) then (q,y) G 7r* artja |(Goal). Set W = 7r* artja |(Goal), by contradiction suppose that 
(q, y) $l W, we will construct a non-winning execution compatible with A. By hypothesis 
^partial (W) = W so (q,y) ^ TTpartial (W) , it follows that for all a G S c , for all r G M + 
there exists (x,i) G V\ X V such that 7 g (x, t) = y, and (q,y) ~^ T xt WiV 1 ) implies (q',y') ^ 
Pred a (W) or Post^ +r j n uPred(TF) ^ 0. Let (r, a) = X(q,y) (as A is a strategy under 

partial observation it does not depend of x and t) and let (x, t) & V± X M + be as in the 
previous statement. 

There exists (gi, xi, ii, with (gi, y{) ^ W such that either (g, x, t, y) (gi, x\, t\, yx) 

or there exists r' ^ r and it G S u with (q,x,t,y) — ^> (gi, xi, ti, j/i). In both cases, the 
constructed execution is compatible with A. As (qi,yx) £ W we can repeat the same 
argument and construct inductively an execution p = (q,x,t,y) n ' 1 > (gi, xi, ti, yi) T2 '° 2 > 
. . . compatible with A and such that for every i, (qi,Xi,ti,yi) ^ W. By definition of W, for 
every i, qi ^ Goal, which contradicts the assumption that A is a winning strategy. □ 

7r* artja |(Goal), but this does not imply that we can compute this set, as some Ai- 
structures have an undecidable theory. The following corollary states that if some conditions 
on the structure and on TTpartial are satisfied, then this procedure provides an algorithmic 
solution to the control problem. 

Corollary 5.6. Let M be a structure such that Th(A4) is decidable^ Let C be a class of 
AA-games such that for every A in C, there exists a finite partition V of Q x V2 definable 
in M., respecting GoaQ, and stable under 7Tp ar tj a i u Then the control problem under partial 
observation in the class C is decidable. Moreover if A G C, the set of winning states under 
partial observation of A is computable. 

Proof. Let M be a structure and C a class of automata satisfying the hypotheses and take 
A G C. As V is stable under 7r part j a |, 7r* artia |(Goal) is a finite union of pieces of V. Hence 
there exists n G N such that 7r* artia |(Goal) = 7Tp artia |(Goal). Thus proposition 15.51 shows that 
the set of winning states is 7r* artja |(Goal). 
As 

^partial an d Goal are definable, we have that 7rp artja |(Goal) is definable and as Th(A4) 
is decidable we can test if 7Tp artia |(Goal) = 7r p g rt j a |(Goal), we can thus effectively find a 
representation of 7r* artja |(Goal). 

As Th(A4) is decidable, if a state (q, y) is definable we can test if (q, y) G 7r* artia |(Goal). 
It follows that the control problem in an A4-structure is decidable. □ 



5.2. Superwords and the vrp art j a | Operator. We now present a sufficient condition for 
a partition to be stable under the operator 7r pa rtial : we require that the partition is stable 
under Pred a (for all a G S) to handle the discrete part of the automaton and we show that 
the stability by superwords is fine enough to be correct for solving control problems under 
partial observation. 

^We recall that a theory Jh(M) is decidable iff there is an algorithm which can determine whether or not 
any sentence (i.e., a formula with no free variable.) is a member of the theory (i.e., is true). We suggest to 
readers interested in general decidability issues on o-minimal hybrid systems to refer to Section 5 of BM05 . 
I.e., Goal is a union of pieces of V '. 

^Meaning that if P is a piece of V then 7r par ti a i(P) is a union of pieces of V. 
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Problem 5.7. Let A be an M-g&me and V be a partition of Q x Vi- If V respects Goal, 
is stable under Pred a (for all a € E) and superword-stable, then V is stable under the 
operator 7r partia |. 

Proof. We fix a location g of the automaton and we take 2/1,2/2 £ V2 such that there exists 
i E ? with 2/1,1/2 £ A We now show that if 2/1 £ ^partial P0> f° r some X € P then 
2/2 £ ^partial (A). I n case 2/1 £ then X = A and 2/2 £ Y ■ 

We assume 2/1 £ ^partial (X) \ A. There exists a € S c and n £ M + such that for 
all (x,t) € Vi x V with r j q {x,t) = 2/1 and for all 2/1 such that 2/1 - ^a;,t 2/i> we have that 
y'l £ Pred a (X), and Post? t '^ +r i C uPred(A). Let us now express the previous condition in 
term of superword. Assume that 

Supp(yi) = S1S2 ■ ■ ■ Sk, where Si G 2 V , 
the previous condition means that Supp(2/i) contains a prefix Si - ■■ Si is such that: 

• for all Pi € Si, we have that Pi C Pred a (A) (this condition makes sense since P 
is stable under Pred a ; indeed, a priori we only have that there exists y[ € Pj such 
that y[ £ Pred a (X), the stability of P under Pred a implies that P C Pred a (X)), 

• for all j ^ I, for all Pj € Sj, we have that uPred(X) n Pj = (again this condition 
makes sense since P is stable under Pred a ). 

Since V = Sup (V) and both 2/1 and 2/2 belong to the same piece of V, we have that 
Supp(2/i) = Sup^(2/2) = S1S2 • • • Sk- In particular, we can find T2 £ M + such that if 
2/2 — > 2/2; we have that y' 2 corresponds to the letter Si. Thus we have that y' 2 £ Pred a (X) 
and P0Stj^ +r2] C uPred(X), i.e. 2/2 £ vr par tiai(X). □ 

As an immediate corollary of this proposition and of Corollary 15.61 we get the following 
general decidability result. 

Corollary 5.8. Let M be a structure such that Th(A4) is decidable. Let C be a class of 
Ad-games such that for every A in C, there exists a finite partition V ofQxVz definable in 
A4, respecting Goal, superword-stable, and stable under Pred a for every action a G S. Then 
the control problem under partial observation (Problem \3.1'J\) in the class C is decidable, and 
if A £ C, the set of winning states under partial observation of A is computable. 

5.3. A Note on the Perfect Observation Framework. We briefly discuss the perfect 
observation framework. We have already seen that it is a special case of the partial obser- 
vation framework (see Proposition 13.16] ). Hence, we can reuse the previous results and get 
decidability and computability results. However, we can also define an appropriate control- 
lable predecessor operator 7Tp er f ect that will be correct in the perfect observation framework. 
The new operator 7r per f ect is just a twist of the previous operator, which we define as: 

TTperfectW = W u time-Pred pe rfect (cPred(HO, uPred(W)) 

where time-Pred per f ec t existentially quantifies on pairs (x,t) such that y = -y q (x,t) (instead 
of universally quantifying on those pairs, as in time-Pred part j a i). 

Remark 5.9. In the perfect observation framework, the controller is aware of the precise 
trajectory that will be followed, hence his choice of action can be done after his choice of 
delay contrarily to the partial observation case (remember Remark 15. 3h . That is why the 
union over actions is put within the scope of the safe time predecessor in vr per f ec t. 
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Applying similar reasoning as in the previous sections, we can prove that 7r* erfect (Goal) 
corresponds to the set of winning states of A, and that a partition, which is both stable 
under Pred a (for every a € S) and suffix-stable, is actually correct for solving control 
problems in the perfect observation framework. We can thus state the following theorem. 

Theorem 5.10. Let M be a structure such that Th(A4) is decidable. Let C be a class of 
M- games such that for every A in C, there exists a finite partition V of Q x V2 definable 
in A4, respecting Goal, suffix- stable, and stable under Pred a for every action a € E. Then 
the control problem under perfect observation (Problem \3.8\) in the class C is decidable, and 
if A € C, the set of winning states under perfect observation of A is computable. 

Note that being suffix-stable is a stronger condition than being a time-abstract bisim- 
ulation [Bri07], and we see here that this is one of the right tools to solve control problems. 
For instance in Example 13.181 the partition V is a time-abstract bisimulation but is not 
suffix-stable. Indeed s\,s% £ A but Sufp(si) 7^ Sufp(s2). 

Remark 5.11. Using the results of this section, we recover the results of [AMPS98] about 
control of timed automata. Note that for the timed automata dynamics (remember Ex- 
ample 13.2ft partial or perfect observation do not make a difference (the dynamics is deter- 
ministic). Indeed we consider the classical finite partition of timed automata that induces 
the region graph (see [AD94]). Let us call Vr this partition, and notice that Vr is de- 
finable in (R, <, +, 0, 1). Vr is stable under the action of Pred a for every action a € S. 
By Example 13.21 the continuous dynamics of timed automata is definable in (R, <, +, 0, 1). 
Hence it makes sense to encode continuous trajectories of timed automata as words. Then 
one can easily verify that SuOVr) = Vr. By Theorem 15.101 we get the decidability and 
computability of winning states under perfect information in timed games [AMPS98] as a 
side result. 

Corollary 5.12. The control problem under perfect information in the class of timed au- 
tomata is decidable. Moreover the set of winning states under perfect observation is com- 
putable. 

6. O-Minimal Games 

In this section, we focus on the particular case of o- minimal games (i.e., Al-games 
where A4 is an o-minimal structure and in which extra assumptions are made on the re- 
sets). We first briefly recall definitions and results related to o- minimality [PS86] , We 
show that existence of finite partitions which are stable w.r.t. the controllable predecessor 
operator can be guaranteed for o-minimal games. More precisely, we first show that, in 
this framework, a partition stable under the controllable predecessor operator can easily 
be obtained via the superword abstraction (this is due to the assumptions on the resets). 
Then, we use properties of o-minimality to prove the finiteness of the previously obtained 
partition. Finally we focus on o-minimal structures with a decidable theory in order to ob- 
tain full decidability and computability results. As in the previous section, we mostly focus 
on the partial observation framework, but also mention results in the perfect observation 
framework. 
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6.1. O-Minimality. We recall here the definition of o-minimality and the " Uniform Finite- 
ness Theorem" that will be applied later in this section. The reader interested in o- 
minimality should refer to [vdD98j for further results and an extensive bibliography on 
this subject. 

Definition 6.1. An extension of an ordered structure M = (M, <,...) is o-minimal if every 
definable subset of M is a finite union of points and open intervals (possibly unbounded). 

In other words the definable subsets of M are the simplest possible: the ones which are 
definable in (M, <). This assumption implies that definable subsets of M n (in the sense of 
A4) admit very nice structure theorems (like the cell decomposition [KPS86]) or Theorem l6.2l 
below. The following are examples of o-minimal structures: the ordered group of rationals 
(Q, <, +, 0, 1), the ordered field of reals (R, <, +, •, 0, 1), the field of reals with exponential 
function, the field of reals expanded by restricted pfaffian functions and the exponential 
function, and many more interesting structures (see [vdD98, Wil96 ). An example of non 
o-minimal structure is given by (R, <,sin, 0), since the definable set {x | sin(x) = 0} is not 
a finite union of points and open intervals. However, let us mention that the structural 
(R,+, -,0, 1, <,sin|. ,coS|. ) is o-minimal (see |vdD96] ). 

Theorem 6.2 (Uniform Finiteness jKPS86j ). Let M = (M, <,...) be an o-minimal struc- 
ture. Let S C M m x M n be definable (in M ), we denote by S a the fiber {y G M n \(a, y) G S). 
Then there is a number N$ G N such that for each a G M m the set S a C M n has at most 
Ns definably connected components. 

6.2. Generalities on O-Minimal Games. 

Definition 6.3. Given A an .M-game, we say that A is an o-minimal game if the structure 
M is o-minimal and if all transitions (g, <?, a, R, q') of A belong tc0 Q x 2 V2 x S x 2 V2 x Q. 

Let us notice that the previous definition implies that given A an o-minimal game, the 
guards, the resets and the dynamics are definable in the underlying o-minimal structure. 
We denote by Va the coarsest partition of the state space S = Q x V 2 which respects Goal, 
and all guards and resets in A. Note that V_a is a finite definable partition of S. 

Due to the strong reset condition we have that Va is stable under the action of Pred a 
for every action a. This holds by the same argument that allows to decouple the continuous 
and discrete components of a hybrid system in [LPS00]. Let us also notice that, in the 
framework of o-minimal games, any refinement of V_a is stable under the action of Pred a 
for every a G X. 

Example 6.4. The continuous dynamics of timed automata (see Example 14. 4ft is definable 
in the o-minimal structure (M, +, 0, 1, <}. The continuous dynamics of rectangular automata 
(see Example I4.5P is definable in the o-minimal structure (M, +, •, 0, 1, <). Hence games on 
timed (resp. rectangular) automata with strong resets are particular cases of o-minimal 
games. The .M-game of Example 13.101 is in fact an o-minimal game; indeed one can see 
that it can be defined in the structure (R, +, •, 0, 1, <,sin| 2jr] , cos| [0 ) which is o-minimal 
(see |vdD96| K 



'sin | [0 27r] and cos| [Q 2?r] correspond to the sinus and cosinus functions restricted to the segment [0, 2-k\. 
^This is a particular case of reset for .M-game where we consider only constant functions for resets. 
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6.3. Solving O-Minimal Games. In this subsection, we will see how we can (easily) 
build a partition which is stable under the actions of the controllable predecessor operator. 
The key ingredients to build this partition will be (i) the strong resets conditions and (ii) 
the superword abstraction. The finiteness of the obtained partition will be discussed in 
Subsection 16.41 

Problem 6.5. Let A be an o-minimal game, and V_a the partition corresponding to its 
guards and resets. The superword (resp. suffix) partition Sup(7 , ^4) (resp. Suf('P^)) is 
stable under the action of TTpartial (resp. 7Tp er f ect ). 

Proof. This proposition is not a corollary of Proposition 15.71 as Sup^^) is not superword- 
stable. However, the proof of Proposition 15.71 only relied on the fact that in a superword- 
stable partition, two points in a piece of the partition have the same superword abstraction, 
which is precisely what we have in the current case. Hence the previous proof can be 
mimicked, and we do not write all details. It is worth noting also that we do not use all 
properties of o-minimal games, but only the strong reset property, which ensures that the 
partition is stable under Pred a for every action a G S. □ 

6.4. Definability and Finiteness Issues. In the previous subsection, we have proved 
that, given A an o-minimal game, the partition Sup(V_4,) (resp. Sui(VjC)) is stable under the 
action of the controllable predecessor operator under the partial (resp. perfect) observation 
framework. We will now show that this partition is finite. For this we will exploit the 
finiteness property of o-minimality and in order to do so, we first need to prove that our 
encodings are definable. 

6.4.1. Definability. Let (M,*y) be a dynamical system and V be a finite partition of V%. 
We now would like to show that in the case of o-minimal dynamical system the superword 
encoding previously discussed can be done in a definable way. The approach closely follows 
the one used in [Bri06l Section 12.2] for the suffix abstraction (called suffix dynamical type 
in this paper). 

Let (A^,7) be an o-minimal dynamical system and V he a finite definable partition of 
V2. First let us notice that, since V is finite and definable, given one can easily write 

a first-order formula f(y,r) which is true if and only if T y {r) = S (where T y is defined 
similarly to T x - see page [1~1~|) . Let us give this formula, assuming that S = {A\, . . . , A n }: 



Thus, for each y G V2, the set T y exactly consists of the connected components of the sets 
{r G M + I ips(y, t)}, for S G 2 T ' '; i.e. T y is a set of intervals. In order to show that T y is 
first-order definable we need to encode each interval I C M as a point in some cartesian 
power of M. An interval / C M is entirely characterized by (i) its end-points and (ii) the 
fact of being right (resp. left) open or closed. For (i) we formally need a couple to represent 
a single end point in order to recover —00 and +00 (as in the projective line case). For 
(ii) we can use a binary encoding, let us say means open and 1 closed. Thus any interval 
I C M will be encoded by an element (a\, 02, 03, b\, 62, 63) G M 6 . For instance, the interval 
/ = {x G M I x ^ 5} is encoded by (5,1,1,1,0,0). Thanks to this "trick", one can find a 




A Vx Vt (j(x,t) 



y) (7(x,t + r) G A 1 U---UA n ). 
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first-order formula tp y defining T y . The writing of the formula <p y is not difficult but rather 
tedious: different cases have to be considered (depending on whether the interval /, encoded 
by an element of M 6 , is left (resp. right) bounded and left (resp. right) open or closed). 
Further details of the construction of the formula can be found in [Bri06, Section 12.2]. 

6.4.2. Finiteness. We will now prove that when considering o-minimal dynamical systems, 
only finitely many finite superwords are needed to encode all possible trajectories. 

Problem 6.6. Let (.M,7) be an o-minimal dynamical system and V be a finite definable 
partition of V2. There exists finitely many finite superwords associated with (A4,~f) w.r.t. 
V. 

Proof. Given S G 2^ let us first consider the set 

T y (S) = {t£ M + I T y {r) = S) = {t g M+ | <p s (y,r)}. 

By the above discussion, the set J-y(S) is a definable subset of M. Hence by o-minimality 
it is a finite union of points and open intervals, in particular, it has only finitely many 
connected components. By definition of J- y we have the following equality. 

\F y \ = ^2 (^number of connected components of T y (S)j . 

SG2^ 

Since V is finite we can conclude that T y is finite. 

Using the uniform finiteness theorem (Theorem 16. 2p we obtain that there exists A'Gff 
such that for all y G V2 we have that \j- y \ ^ N . 

In terms of word encoding, this means that there are only finitely many superwords 
associated with the points of the (output) space V2. More precisely, the superwords Supp(y) 
have lengths uniformly bounded by N. Since the superwords Sup-p(y) are words on the 
finite alphabet 2^, this completes the proof. □ 

The previous proposition directly implies the finiteness of the partition Sup('P). More- 
over we have that this partition is definable, as stated in the following proposition. 

Problem 6.7. Let (M,^) be an o-minimal dynamical system, V be a finite definable 
partition of the output space Vi- The partition Sup('P) is finite and definable. 

Proof. Since there are only finitely many superwords, it suffices to show that given y G V2 
and SW a superword on V (i.e. a word on 2^), we can define (by a first-order formula) 
that SW = Sup-p(y). Suppose that SW = Si ■ ■ ■ S k ■ ■ ■ S n , where S k G 2 P . We have that 
SW = Sup-p(y) if and only if the following formula holds. 

3n G M+ 3r 2 G M + , • • • 3r n G M + , 3h G F y , I 2 € T y , ■■■ 3I n G T y 

n 

{n <T 2 < ■■■ <T n ) A f\ T y {T k ) = S k A Fy = {h, I 2 , ■ ■ ■ , In}- 

k=l 

Notice that the above formula is first-order since T y is first-order definable and testing 
whether F y {T k ) = S k is also first-order definable. □ 
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6.5. Synthesis of Winning Strategies. We now prove that given A an o-minimal game 
definable in A4, we can construct a definable strategy (in the same structure A4) for the 
winning states under partial observation. The effectiveness of this construction will be 
discussed later. 

Theorem 6.8. Given A an o-minimal game, there exists a definable memoryless win- 
ning strategy under partial (resp. perfect) observation for each (q,y) G 7r* art j ai (Goal) (resp. 

perfect ( Goal );- 

Proof. By Proposition l6.5l the partition Sup(V^) is finite, definable and stable under vr part j a |. 
In particular, there exists thus n G N such that 7r* artia |(Goal) = 7Tp artia |(Goal). Hence, by 
Proposition 15.51 7rp artia |(Goal) is the set of winning states. 

Given (q, y) G 7rp artia |(Goal), we know that there exists a winning strategy from (q,y). 
We now have to point out a definable winning strategy from (q,y). Following the proof 
of Proposition 15.51 we build the definable strategy by induction on the number of it- 
erations of vTp art j a |. Let us suppose we have already built a strategy on each piece of 

W = 7Tp artia |(Goal), let us now consider 7r part j a |(VF) \ W. 

By Proposition 16.51 we know that 7Tp art j a |(VF) \ W is a finite union of pieces of Sup('Pyi). 
Let P be one of these pieces. We know that P corresponds to a finite superword on V^. 
Thus given (q, y) G P we have that 

Sup^ (y) = SiS 2 • • • S k , where S l G 1 Va . 

Since (q, y) G TTpartial (^^) \ the superword Supj, A (y) contains a prefix S\ ■ ■ ■ Si such 
that there is a G T, c with: 

• for all P l £S h P i c Pred a (W), _ 

• for all j sC I, for all P { G Sj, uPred(W) n P; = 0. 

Since for all Pi G Si, we have that Pi C Pred a (W), the controllable action a G S c is such 
that given any (q, y) G Si a transition labelled by a is enabled and all such transitions lead 
to W. The strategy for (q,y) will be to perform action a after some delay. We now explain 
how to choose this delay. 

Let (q, y) be such that (q,y) G P. Let us consider Time(y) the subset of M + defined 
as follows: 

Time(y) = {r G M + \ By' G Si such that (q,y) ^ (q,y')}- 
This set is definable since Si is definable. 

By o-minimality, we have that Time(y) is a finite union of points and open intervals. 
Let us denote by / the leftmost point or interval. Let us notice that / is definable. If / has 
a minimum m, we define X(q,y) = (m,c). Otherwise two cases may occur. If / is bounded 
then it is of the form (m,m') or (m, m ] in this case we defined X(q,y) = {\{ra + m'),c). 
Finally if / has no minimum and is unbounded it is of the form (m, oo) and in this case we 



"^Let us recall that every o-minimal ordered group is torsion free and divisible (see [PS86] ). this implies 
there exists a unique y satisfying y + y = (m + m'), which we note \ira + m'). 
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define X(q,y) = (to + 1, c). We summarized the definition of A on Si as follows: 

(min(J),c) if (p x (y) 

Kq, y) = \ (h( inf i 1 ) + s Mi)) , c) if My) 

(inf(J) + 1, c) otherwise 

where <pi(y) is a formula which is true if and only if / (or Time(y)) has a minimum and 
f2(y) is a formula which is true if and only if / has no minimum and is bounded. Thus 
clearly A is definable. 

Since there are finitely many P € Sup(Vj,), we can conclude that A is definable. □ 

Remark 6.9. Note that the memoryless strategy given by Theorem 16.81 is computable if 
partial ( Goal ) is - 

Remark 6.10. Let us notice that in the case of timed automata dynamics (described 
in Example 13. 2p , our definable strategies correspond to the realizable strategies computed 
in [BUFL04j . 



6.6. Decidability Result. Theorem 16.81 is an existential result. It claims that given an o- 
minimal game, there exists a definable memoryless strategy for each y € 7r* artja |(Goal), and 
by Theorem 16.51 we know that Sup(7 3 ^) is finite. The conclusion of the previous subsection 
is that given an o-minimal game there exists a definable memoryless winning strategy for 
each y 6 vr* artia| (Goal). 

In general, Theorem 16.81 does not allow to conclude that the control problem in an 
.M-structure is decidable. Indeed it depends on the decidability of Th(A4). We can state 
the following theorem: 

Theorem 6.11. Let M. be an o-minimal structure such that Th(7W) is decidable and C 
a class of Ad-automata. Then the control problem under partial (resp. perfect) observa- 
tion in class C is decidable. Moreover if A € C, the set of winning states ^p ar tj a | (Goal) 
(resp. 7r* erfect (Goal)j under partial (resp. perfect) observation is computable and a mem- 
oryless winning strategy can be effectively computed for each (q,y) £ 7r* artja |(Goal) (resp. 

perfect ( Goal );- 

Proof. By Proposition ^. 7| for each A € C, Sup(VjC) is a definable finite partition respecting 
Goal. Moreover by Proposition 16. 5| Sup('P^) is stable under vrp art j a i- Hypothesis of Corol- 
lary 15.61 are thus satisfied and we get that the control problem in class C is decidable and 
that the winning states of a game A € C are computable. Moreover Theorem 16.81 ensures 
that a memoryless strategy can be effectively defined from such winning states. □ 



12 Let us notice that the way we extract a single point from Time(j/) is nothing more than the curve 
selection for o-minimal expansions of ordered abelian groups, see [vdD98l chap. 6]. 
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Remark 6.12. (R, <,+,0, 1) and (R, <,+,-, 0, 1) are examples of o-minimal structures 
with decidable theory and so o-minimal games based on theses structures can be solved by 
Theorem 16. Ill 

Remark 6.13. In this paper we did not distinguish Zeno behaviours. In particular, in our 
framework, if the environment has a strategy that prevents the game to reach the Goal 
locations by blocking time, we say that the controller loses the game. In the framework 
of timed automata, an ad-hoc solution to this problem of Zenoness has been proposed 
in AFH + 03 . However, due to the strong reset conditions of o-minimal hybrid systems, 
the method of [AFH + 03] cannot be easily applied to our framework, but this problem is 
somehow orthogonal to ours. 



7. Conclusion 

In this paper we have studied games based on dynamical systems with general dynamics, 
both under a prefect and a partial observation of the dynamics. Under the first hypothesis, 
we have shown that time-abstract bisimulation is not fine enough to solve these games, 
which is a major difference with the case of timed automata. By means of an encoding 
of trajectories by words, we have obtained a good abstraction for control problems (with 
reachability winning conditions, but it applies also to basic safety winning conditions). We 
have finally provided decidability and computability results for o-minimal games under both 
perfect and partial observation hypothesis. Our technique applies to timed automata, and 
we recover decidability of timed games [AMPS98], as well as the construction of winning 
strategies [BCFL 04] as side results. 
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